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Abstract.  Computer  programming  of  complex  systems  is  a time  consuming 
effort.  Results  are  often  brittle  and  inflexible.  Evolving,  self-learning  flexible 
multi-agent  systems  remain  a distant  goal.  This  paper  analyzes  difficulties 
toward  developing  evolving  systems  and  proposes  new  solutions.  The  new 
solutions  are  inspired  by  our  knowledge  of  the  human  mind.  The  mind  develops 
language  and  cognitive  abilities  jointly.  Real-time  sensor  signals  and  language 
signals  are  integrated  seamlessly,  before  signals  are  understood,  at  pre- 
conceptual  level.  Learning  of  conceptual  contents  of  the  surrounding  world 
depends  on  language  and  vice  versa.  This  ability  for  integrated  communication 
and  cognition  is  a foundation  for  evolving  systems.  The  paper  describes  a 
mathematical  technique  for  such  integration:  fuzzy  dynamic  logic  and  dual 
cognitive-language  models.  We  briefly  discuss  relationships  between  the 
proposed  mathematical  technique,  working  of  the  mind,  applications  to 
understanding-based  search  engines  and  evolving  multi-agent  systems. 


1 Cultural  Evolution:  Computers  Versus  Nature 

Computer  capabilities  for  communication  and  cognition  currently  are  developed 
separately,  usually  in  different  organizations.  The  nature  does  it  differently.  A child 
develops  both  capabilities  jointly.  We  do  not  know  if  it  is  possible  to  code  computers 
to  be  ‘cognitive’  or  ‘language  capable’,  one  capability  separately  from  the  other. 
Current  engineering  approaches  could  be  invalid  in  principle.  These  considerations 
are  prime  motivations  for  this  paper.  Let  us  examine  them  in  some  details.  Evolution 
of  the  human  mind  from  pre-human  ancestors  occurred  in  three  stages:  genetic 
evolution,  cultural  evolution,  and  learning  along  with  ontological  development  of  an 
individual  child  into  an  adult.  Cultural  evolution  and  cognitive  learning  are  much 
faster  than  genetic  evolution.  This  paper  concentrates  on  the  mechanisms  of  cultural 
evolution  and  learning. 

As  physical  infrastructure  for  communication  systems  and  the  Internet  matures,  the 
information  services  are  gaining  in  importance.  Distributed  integration  of  sensor 
signals  with  flexible  communications,  with  data  and  text  data  bases  would  be 
necessary  for  the  future  Sensor  web,  an  integrated  operation  of  multiple  users  and 
agents  using  sensors  and  communications.  However,  computer  systems  today  use 
inflexible  models  and  ontologies.  Communication  systems  use  fixed  protocols. 
Contents  of  communications  are  intended  for  human  understanding,  computers  do  not 
understand  contents. 
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Practical  implementations  of  complex  multi-agent  systems  with  communication 
and  cognitive  abilities  are  based  on  detailed  models  and  protocols.  These  systems  lack 
the  flexibility  of  human  cognition  and  natural  languages. They  integrate  signals  from 
sensors  and  communication  messages  only  at  high  cognitive  levels  of  logical 
predicates.  First,  information  has  to  be  extracted  from  sensor  signals  and  formulated 
as  logical  statements  at  the  appropriately  high  level  of  abstraction.  Similar  language 
or  communication  messages  have  to  be  pre-processed,  the  relevant  data  extracted  and 
formulated  as  logical  statements  at  a similar  level  of  abstraction.  Integration  rely  on 
models,  ontologies,  and  protocols,  which  assume  shared  knowledge  and 
understanding  [1].  In  practice,  structures  of  these  models  have  to  be  fixed.  This  is  also 
true  for  ontologies  being  developed  for  semantic  web.  They  are  not  as  flexible  as 
“shared  knowledge”  necessary  for  understanding  among  people.  Specific 
mathematical  reason  for  this  inflexibility  we  discuss  in  section  2.  The  resulting 
systems  are  brittle.  As  requirements  and  hardware  are  changing,  they  become 
obsolete. 

Contrary  to  the  brittleness  of  artificial  fusion  systems,  the  human  mind  improves 
with  experience.  We  discuss  in  this  paper  that  learning,  adaptive,  and  self-evolving 
capabilities  of  the  mind  are  closely  related  to  the  ability  to  integrate  signals 
subliminally.  For  example,  during  everyday  conversations,  human  eye  gaze  as  well  as 
visual  processing  stream  and  the  type  of  conceptual  information  extracted  from  the 
surrounding  world  are  affected  by  contents  of  speech,  even  before  it  is  fully  processed 
and  conceptually  analyzed.  Similarly,  speech  perception  is  affected  by  concurrent 
cognitive  processing.  To  some  extent,  we  see,  what  we  expect  to  see;  verbal 
preconditioning  affects  cognition,  and  vice  versa.  This  close,  pre-conceptual 
integration  of  language  and  cognition  is  important  not  only  in  real-time  perception 
and  cognition,  but  also  in  ontogenesis,  during  child  growing  up,  as  well  as  in 
evolution  of  culture  and  language.  Concepts  used  by  individual  minds  evolve  over 
generations  in  interaction  among  multiple  agent-minds.  As  we  attempt  to  develop 
intelligent  systems,  these  lessons  from  biological  systems  and  their  evolution  should 
be  taken  into  account. 

Developing  integrated  systems  with  language  and  cognition  abilities  might  seem 
premature.  Even  considered  separately,  these  problems  are  very  complex  and  far  from 
being  solved.  Our  systems  for  recognition,  tracking,  and  fusion  using  sensor  data 
often  fall  far  short  of  human  abilities.  Similarly,  our  computer  communication 
systems  lack  flexibility  of  language.  Natural  language  understanding  remains  a distant 
goal.  Let  me  repeat  that  the  only  way  two  computers  can  communicate  at  all,  is  due  to 
fixed  protocols.  Communications  among  computers  are  intended  for  human  users. 
Computers  do  not  understand  contents  of  communication  messages,  except  within 
narrow  domains.  Everyone  knows  frustration  of  searching  information  on  the  Internet; 
Google  and  Yahoo  do  not  understand  our  language.  But,  why  should  we  hope  to 
achieve  progress  in  fusing  two  capabilities,  neither  of  which  is  at  hand? 

The  answer  was  given  at  the  beginning  of  the  paper.  The  only  system  that  we  know 
capable  of  human  level  cognition  and  communication  is  the  human  mind.  An 
individual  human  mind  develops  both  capabilities  in  ontogenesis,  during  childhood, 
jointly.  This  is  opposite  to  current  engineering  approaches,  which  attempt  to  develop 
these  capabilities  separately,  usually  in  different  scientific  and  engineering 
organizations.  It  is  quite  possible  that  coding  a computer  to  acquire  language  and 
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cognitive  abilities  similarly  to  the  human  ways  is  an  ‘easier’  task,  possibly,  the  only 
way  to  go.  We  do  not  even  know  if  it  is  possible  to  code  computers  to  be  ‘cognitive’ 
or  ‘language  capable’,  one  capability  separately  from  the  other.  These  current 
approaches  could  be  invalid  in  principle. 

A similar  argument  is  applicable  to  the  ‘initial’  computer  code,  which  we  would 
like  to  be  similar  to  inborn  child’s  capabilities,  enabling  joint  learning  of  language 
and  cognition.  Human  evolved  this  capability  over  at  least  two  million  years.  It  is 
possible,  that  simulating  an  accelerated  evolution  is  an  ‘easier’  scientific  and 
engineering  approach,  than  ‘direct  coding’  into  a computer  of  the  current  state  of 
human  baby  mind.  Moreover,  we  do  not  need  to  have  to  simulate  evolution  of  culture; 
computers  may  learn  from  humans  in  collaborative  human-computer  environment. 
Therefore,  along  with  smart  heuristic  solutions,  we  should  try  to  uncover  natural 
mechanisms  of  evolving  language  and  culture,  and  to  develop  mathematical 
descriptions  for  these  processes. 

Scientific  understanding  of  relationships  between  language  and  cognition  in  the 
past  went  through  several  reversals.  Close  relationships  between  language  and 
cognition  encouraged  equating  these  abilities  in  the  past.  Rule-based  systems  and 
mathematics  of  logic  implied  significant  similarities  between  the  two;  Thoughts, 
words,  and  phrases,  all  are  logical  statements.  The  situation  has  changed,  in  part  due 
to  the  fact  that  logic-rule  systems  have  not  been  sufficiently  powerful  to  explain 
cognition,  nor  language  abilities,  and  in  part  due  to  improved  scientific  understanding 
(psychological,  cognitive,  neural,  linguistic)  of  the  mechanisms  involved.  Many 
contemporary  linguists  consider  language  and  cognition  to  be  distinct  and  different 
abilities  of  the  mind  [see  [2]  for  further  references]. 

Language  mechanisms  of  our  mind  include  abilities  to  acquire  a large  vocabulary, 
rules  of  grammar,  and  to  use  the  finite  set  of  words  and  rules  to  generate  virtually 
infinite  number  of  phrases  and  sentences  [3,4].  Cognition  includes  abilities  to 
understand  the  surrounding  world  in  terms  of  objects,  their  relationships  (scenes  and 
situations),  relationships  among  relationships,  and  so  on  [5].  Researchers  in 
computational  linguistics,  mathematics  of  intelligence  and  neural  networks,  cognitive 
science,  neuro-physiology  and  psychology  during  the  last  twenty  years  significantly 
advanced  understanding  of  the  mechanisms  of  the  mind  involved  in  learning  and 
using  language,  mechanisms  of  perception  and  cognition  [3,4,5,6,7,8].  Much  less 
advance  was  achieved  toward  deciphering  mechanisms  relating  linguistic  competence 
to  cognition  and  understanding  the  world.  Although  it  seems  clear  that  language  and 
cognition  are  closely  related  abilities,  intertwined  in  evolution,  ontogenesis,  and 
everyday  use,  still  the  currently  understood  mechanisms  of  language  are  mainly 
limited  to  relations  of  words  to  other  words  and  phrases,  but  not  to  the  objects  in  the 
surrounding  world,  not  to  cognition  and  thinking.  Possible  mathematical  approaches 
toward  integrating  language  and  cognition,  words  and  objects,  phrases  and  situations 
are  discussed  in  this  paper.  This  might  be  a foundation  for  cognitive  learning  and 
mechanisms  of  cultural  evolution. 

The  paper  starts  with  a mathematical  description  of  cognition,  which  still  is  an 
issue  of  much  controversy.  Among  researchers  in  mathematical  intelligence  it  has 
become  appreciated,  especially  during  the  last  decades  that  cognition  is  not  just  a 
chain  of  logical  inferences  [5,8].  Yet,  mathematical  methods  describing  cognition  as 
processes  in  human  mind  involving  concepts,  instincts,  emotions,  memory. 
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imagination  are  not  well  known,  although  significant  progress  in  this  direction  was 
achieved  [5,8],  A brief  historical  overview  of  this  area  including  difficulties  and 
controversies  is  given  in  the  next  two  sections  from  mathematical,  psychological,  and 
neural  standpoints.  It  is  followed  by  a mathematical  description  of  cognitive 
processes,  including  image  recognition,  tracking,  and  fusion  as  variations  of  the  same 
basic  paradigm.  Then  the  paper  discusses  the  ways  in  which  the  mathematical 
description  of  cognition  can  be  combined  with  language,  taking  advantage  of  recent 
progress  in  computational  linguistics.  It  touches  upon  novel  ideas  of  computational 
semiotics  relating  language  and  cognition  through  signs  and  symbols.  Approaches  to 
understanding-based  web  mining  and  building  integrated  multi-agent  systems  are 
discussed. 

In  conclusion,  I briefly  touch  on  relationships  between  mathematical, 
psychological,  and  neural  descriptions  of  cognitive  processes  and  language  as  parts  of 
the  mind.  It  turns  out  that,  far  from  being  esoteric  abilities  far  removed  from 
engineering  applications,  these  abilities  are  inseparable  from  a mathematical 
description  of  even  simplest  cognition  processes.  Their  understanding  is  helpful  for 
developing  integrated  multi-agent  systems. 


2 Theories  of  the  Mind  and  Combinatorial  Complexity 

Understanding  signals  coming  from  sensory  organs  involves  associating  subsets  of 
signals  corresponding  to  particular  objects  with  internal  representations  of  these 
objects.  This  leads  to  recognition  of  the  objects.  Developing  mathematical 
descriptions  of  this  very  first  recognition  step  was  not  easy;  a number  of  difficulties 
were  encountered  during  the  past  fifty  years.  These  difficulties  were  summarized 
under  the  notion  of  combinatorial  complexity  (CC)  [9].  CC  refers  to  multiple 
combinations  of  various  elements  in  a complex  system;  recognition  of  a scene  often 
requires  concurrent  recognition  of  its  multiple  elements  that  could  be  encountered  in 
various  combinations.  CC  is  prohibitive  because  the  number  of  combinations  is  very 
large:  for  example,  consider  100  elements  (not  too  large  a number);  the  number  of 
combinations  of  100  elements  is  lOO''’®,  exceeding  the  number  of  elementary  particles 
in  a Universe;  no  computer  would  ever  be  able  to  compute  that  many  combinations. 

The  problem  was  first  identified  in  pattern  recognition  and  classification  research 
in  the  1960s  and  was  named  “the  curse  of  dimensionality”  [10].  The  following  forty 
years  of  developing  intelligent  systems  faced  CC  in  various  forms.  Self-learning 
systems  encountered  CC  of  learning  requirements.  Logic-rule  AI  systems  [11]  and 
the  first  Chomsky  ideas  concerning  mechanisms  of  language  grammar  related  to  deep 
structure  [12]  encountered  CC  of  rules.  Model-based  systems  were  proposed  to 
combine  advantages  of  adaptivity  and  learning  with  rules  by  utilizing  adaptive 
models.  Along  similar  lines  were  rules  and  parameters  ideas  of  Chomsky  [13]. 
Model-based  systems  encountered  computational  CC  (N  and  NP  complete 
algorithms).  The  CC  became  a ubiquitous  feature  of  intelligent  algorithms  and 
seemingly,  a fundamental  mathematical  limitation. 

CC  was  related  to  the  type  of  logic,  underlying  various  algorithms  and  neural 
networks  [9].  CC  of  algorithms  based  on  logic  was  related  to  the  Gddel  theory:  It  is  a 
finite  system  manifestation  of  the  incompleteness  of  logic  [14],  Multivalued  logic  and 
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fuzzy  logic  were  proposed  to  overcome  limitations  related  to  the  law  of  excluded  third 
[15].  Yet  the  mathematics  of  multivalued  logic  is  no  different  in  principle  from  formal 
logic.  Fuzzy  logic  encountered  a difficulty  related  to  the  degree  of  fuzziness:  If  too 
much  fuzziness  is  specified,  the  solution  does  not  achieve  a needed  accuracy,  if  too 
little,  it  becomes  similar  to  formal  logic. 

Various  approaches  to  signal  and  communication  integration  are  related  to 
mathematical  methods  considered  above.  For  example,  an  influential  and  general 
method  of  Multiple  Hypothesis  Testing  (MHT)  is  a model-based  method.  Its 
combinatorial  complexity  is  widely  appreciated.  Combinatorial  complexity  prevents 
these  mathematical  methods  from  achieving  human-like  flexibility  and  adaptivity.  In 
section  4 we  discuss  a biologically  inspired  mathematical  technique,  which 
overcomes  CC.  The  biological  inspirations  for  this  approach  are  briefly  summarized 
in  the  next  section  3. 


3 Mind:  Instincts,  Concepts,  and  Emotions 

Among  fundamental  mechanisms  of  the  mind  are  instincts,  concepts,  emotions,  and 
control  of  behavior.  Instincts  operate  like  internal  sensors:  for  example,  when  a sugar 
level  in  blood  goes  below  a certain  level  an  instinct  “tells  us”  to  eat.  Concepts  are  like 
internal  models  of  the  objects  and  situations;  this  analogy  is  quite  literal,  e.g.,  during 
visual  perception  of  an  object,  an  internal  concept-model  projects  an  image  onto  the 
visual  cortex,  which  is  matched  there  to  an  image  projected  from  retina  (this 
simplified  description  will  be  refined  later).  Emotions  are  neural  signals  connecting 
instinctual  and  conceptual  brain  regions.  Whereas  in  colloquial  usage,  emotions  are 
often  understood  as  facial  expressions,  higher  voice  pitch,  exaggerated  gesticulation, 
these  are  the  outward  signs  of  emotions,  serving  for  communication.  A more 
fundamental  role  of  emotions  within  the  mind  system  is  that  emotional  signals 
evaluate  concepts  for  the  purpose  of  instinct  satisfaction  [16].  This  emotional 
mechanism  described  in  the  next  section  is  crucial  for  breaking  out  of  the  “vicious 
circle”  of  combinatorial  complexity.  Conceptual-emotional  understanding  of  the 
world  leads  to  actions  (or  behavior)  in  the  outside  world  or  within  the  mind.  In  this 
paper  we  describe  only  one  type  of  behavior,  the  behavior  of  learning  that  is 
improving  understanding  and  knowledge  of  the  language  and  world. 


4 Modeling  Field  Theory  (MFT) 

The  mind  involves  a hierarchy  of  multiple  levels  of  concept-models,  from  simple 
perceptual  elements  (like  edges,  or  moving  dots),  to  concept-models  of  objects,  to 
complex  scenes,  etc.  Modeling  field  theory  (MFT)  [5],  summarized  below,  associates 
lower-level  signals  with  higher-level  concept-models;  a result  is  an  understanding  of 
signals  as  concepts.  The  difficulties  of  CC  described  in  Section  2 are  avoided.  It  is 
achieved  by  a new  type  of  logic,  the  fuzzy  dynamic  logic.  MFT  is  a multi-level,  hetero- 
hierarchical  system.  We  start  with  a basic  mechanism  of  interaction  at  a single  level. 

At  each  level,  the  output  signals  are  concepts  recognized  (or  formed)  in  input  signals. 
Input  signals  are  associated  with  concepts  according  to  the  representations-models  and 
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similarity  measures  at  this  level.  In  the  process  of  association-recognition,  models  are 
adapted  for  better  representation  of  the  input  signals;  and  similarity  measures  are 
adapted  so  that  their  fuzziness  is  matched  to  the  model  uncertainty.  The  initial 
uncertainty  of  models  is  high  and  so  is  the  fuzziness  of  the  similarity  measure;  in  the 
process  of  learning  models  become  more  accurate  and  the  similarity  more  crisp,  the 
value  of  the  similarity  measure  increases.  This  is  a mechanism  of  fuzzy  dynamic  logic. 

Input  signals  {X(n)}  are  enumerated  by  n = 1,...  N;  concept-models  h = 1,...  H, 
are  characterized  by  the  models  (representations)  {M^(n)}  of  the  signals  X(n);  each 
model  depends  on  its  parameters  {S^},  Mj^(S^,n).  In  a highly  simplified  description  of 
a visual  cortex,  n enumerates  the  visual  cortex  neurons,  X(n)  are  the  “bottom-up” 
activation  levels  of  these  neurons  coming  from  the  retina  through  visual  nerve,  and 
M|^(n)  are  the  “top-down”  activation  levels  (or  priming)  of  the  visual  cortex  neurons 
from  previously  learned  object-models  [17].  Cognition  process  attempts  to  “match” 
top-down  and  bottom-up  activations  by  selecting  “best”  models  and  their  parameters. 
Computationally,  it  increases  a similarity  measure  between  the  sets  of  models  and 
signals,  L((X(n)},{M,(n)})  [5]. 

L({X},(M})=n  S r(h)l(X(n)IM,(n));  (1) 

neW  hsH 

here,  l(X(n)IM^(n))  (or  simply  l(nlh))  is  a conditional  partial  similarity  between  one 
signal  X(n)  and  one  model  Mjj(n);  (1)  accounts  for  all  possible  combinations  of 
signals  and  models.  Parameters  r(h)  are  proportional  to  the  number  of  signals  {n} 
associated  with  the  model  h.  Maximization  of  similarity  has  the  following 
psychological  and  neurobiological  interpretation;  it  is  an  instinctual  behavior  that 
evolved  with  the  purpose  of  understanding  the  world,  it  is  instinct  for  knowledge. 

Note,  that  (1)  contains  a large  number  of  combinations  of  models  and  signals,  a 
total  of  items;  this  was  a cause  for  the  combinatorial  complexity  of  the  past 
algorithms  discussed  in  section  2.  MFT  solves  this  problem  using  the  mechanism  of 
fuzzy  dynamic  logic  (DL)  [5,18].  The  DL  iteration’s  consist  of  two  steps:  first 
compute  fuzzy  association  variables  f(hln),  then  improve  parameters 


f(hln)  = r(h)  l(nlh)  / ^ r(h')  l(nlh’). 

(2) 

h'£H 

= (i-a)  + a f(hln)[61nl(nlh)/5Mj,]9M'/aSj,, 

(3) 

r(h)  = N,/N;  N,=  ^ f(hln); 

(4) 

n 


Here,  parameter  a determines  the  iteration  step  and  speed  of  convergence  of  the 
MF  system;  can  be  interpreted  as  a number  of  signals  X(n)  associated  with  or 
coming  from  a concept-model  n.  After  step  (3,  4)  the  iterations  returns  to  step  (2)  and 
continues  until  changes  in  parameters  become  negligible  and  similarity  (1)  stop 
increasing).  The  following  theorem  was  proven  [5]. 

Theorem.  Equations  (2)  through  (6)  define  a convergent  dynamic  system  MF  with 
stationary  states  given  by  max|s^jL. 
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In  plain  language  this  means  that  the  above  equations  indeed  result  in  concept- 
models  in  the  “mind”  of  the  MFT  system,  which  are  most  similar  [in  terms  of 
similarity  (1)]  to  the  sensory  data.  Despite  a combinatorially  large  number  of  items  in 
(1),  a computational  complexity  of  the  MF  method  is  relatively  low,  it  is  linear  in  N. 
This  theorem  is  proved  by  demonstrating  that  similarity  (1)  increases  at  each  iteration 
[5],  Psychological  and  neurobiological  interpretation  of  this  fact  is  that  instinct  for 
knowledge  is  satisfied  with  each  iteration;  MFT  system  ‘enjoys’  the  process  of 
convergence  to  better  knowledge. 

Summary  of  the  MF  convergence',  during  an  adaptation  process,  initial  fuzzy  and 
uncertain  models  are  associated  with  structures  in  the  input  signals,  fiizzy  models  are 
getting  more  definite  and  crisp.  The  type,  shape  and  number  of  models  are  selected  so 
that  the  internal  representation  within  the  system  is  similar  to  input  signals:  the  MF 


Fig.  1.  Finding  ‘smile’  and  ‘frown’  patterns  in  noise,  an  example  of  dynamic  logic  operation: 
(a)  tme  ‘smile’  and  ‘frown’  patterns  shown  without  noise;  (b)  actual  image  available  for 
recognition  (signal  is  below  noise,  signal-to-noise  ratio  is  between  -2dB  and  -0.7dB);  (c)  an 
initial  fuzzy  model,  the  fuzziness  corresponds  to  uncertainty  of  knowledge;  (d)  through 
(h)  show  improved  models  at  various  iteration  stages  (total  of  22  iterations).  At  stage  (d)  the 
algorithm  tried  to  fit  the  data  with  more  than  one  model  and  decided,  that  it  needs  three  models 
to  ‘understand’  the  content  of  the  data.  There  are  three  types  of  models:  one  uniform  model 
describing  noise  (it  is  not  shown)  and  a variable  number  of  blob-models  and  parabolic  models, 
which  number,  location  and  curvature  are  estimated  from  the  data.  Until  about  stage  (g)  the 
algorithm  ‘thought’  in  terms  of  simple  blob  models,  at  (g)  and  beyond,  the  algorithm  decided 
that  it  needs  more  complex  parabolic  models  to  describe  the  data.  Iterations  stopped  at  (h), 
when  similarity  (1)  stopped  increasing.  This  example  is  discussed  in  more  details  in  [19] 
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concept-models  represent  structure-objects  in  the  signals.  Computations  describing  this 
process  are  given  by  fuzzy  dynamic  logic  eqs.  (2,  3,  4).  It  is  illustrated  in  Fig.  1 for 
recognition  of  ‘smiles’  and  ‘frowns’  in  the  background  of  a strong  noise.  In  terms  of  the 
mind,  it  describes  an  elementary  cognition  process  involving  instincts,  imagination, 
emotions  and  concepts,  but  before  discussing  this  cognitive-psychological 
interpretations,  lets  us  briefly  look  into  integrating  this  process  with  language. 

By  using  concept-models  with  multiple  sensor  modalities,  a MFT  system  can 
integrate  signals  from  multiple  sensors,  while  adapting  and  improving  internal 
concept- models.  Similarly,  MFT  can  be  used  to  integrate  all  sources  of  information, 
in  particular,  cognition  and  language.  This  requires  linguistic  MFT  models,  which  can 
be  developed  using  known  linguistic  structures  [3,4,6,7,20].  Here,  I briefly  outline  an 
approach  to  the  development  of  MFT  linguistic  models. 

Let  us  discuss  the  development  of  models  of  phrases  from  words  for  the  purpose  of 
text  understanding.  The  input  data,  X(n),  in  this  “phrase-level”  MF  system,  are  word 
strings,  for  simplicity,  of  a fixed  length,  S,  X(n)  = { w„+i,  w„+2  --  w„+s  }.  Here  w„  are 
words  from  a given  dictionary  of  size  K,  W = {wi,  W2...  Wk},  and  n is  the  word 
position  in  a body  of  texts.  A simple  phrase  model  is  a subset  of  words  from  the 
dictionary,  without  any  order  or  rules  of  grammar  (computational  linguists  call  it 
‘bag’  model), 

M^(S^,n)  = { Wh,i,  Wh,2. . . Wh,sl ; (6) 

the  parameters  of  this  model  are  its  words,  M^(Sj^,n)  = = [Wh.i,  Wh,2-..  Wh,s). 

Language  learning  in  this  simplified  context  consists  in  defining  models-concepts- 
phrases  best  characterizing  the  given  body  of  texts  in  terms  of  a similarity  measure. 
Extension  of  DL  to  language  was  considered  in  [21].  It  can  be  applied  to  bag  models 
and  to  other  known  structures  of  natural  languages,  such  as  noun  and  verb  phrases, 
tree  structures,  etc.  [3,4,6,7,20,22,23].  This  accomplishes  the  goal  of  the  language 
acquisition  project  without  combinatorial  complexity.  This  technique  can  be  applied 
to  the  development  of  understanding-based  web  search  engines.  A hierarchical 
language-MFT  is  developed  with  higher  levels  of  the  hierarchy  extending  ‘above’ 
phrases  to  learning  the  contents  of  paragraphs,  pages,  documents. 

Integration  of  language  and  cognition  in  MFT  is  attained  by  characterizing  objects 
and  situations  in  the  world  with  two  types  of  models,  cognitive  and  language  models; 
so  that  in  eqs.  (1)  through  (4)  Mj^fn)  = (Ml^j(n),  M2^2(")K  cognitive  and  M2 

for  language  models.  Indexes  hi  and  h2  innumerate  cognitive  and  language  models; 
but  it  is  not  necessary  to  consider  combinations  of  hi  and  h2,  because  initially  all 
models  are  same,  fuzzy  blobs,  just  placeholders  for  future  knowledge.  The  above 
equations  describe  a single  agent  with  a MFT-mind. 

A MFT  agent  can  be  used  in  a particular  application  alone  or  as  a part  of 
collaborative  multi-agent  environment.  A single  MFT  agent  can  utilize  complex 
adaptive  models,  and  can  learn  from  data  and  signals.  Several  applications  of  this  type 
were  developed  [5,22].  The  more  specific  the  models  are,  the  less  data  will  be 
required  for  learning.  A different  evolutionary  approach  can  start  with  simple  models 
and  develop  complex  models  and  abilities  as  a result  of  evolution  of  a multi-agent 
MFT  system.  Multiple  agents  can  learn  from  their  environment  along  with 
communicating  among  themselves  or  within  a collaborative  environment  with  human. 
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First  steps  toward  the  development  of  evolving  systems  with  cognitive  and 
communicating  multiple  agents  with  a MFT  mind  are  described  in  [24,25]. 

An  integrated  MFT  system  learns  similarly  to  human,  in  parallel  in  three  realms: 
(1)  language  and  cognitive  models  are  learned  jointly,  when  language  data  are  present 
in  association  with  perception  signals,  like  during  mother  talking  to  a baby:  “this  is  a 
car”  (perception-models  and  word-models),  and  like  during  more  complicated 
conversations:  “Look  at  Peter  and  Ann,  they  are  in  love”  (cognitive-models  and 
phrase-models);  (2)  language  models  are  learned  independently  from  cognition,  when 
language  data  are  encountered  for  the  first  time  with  no  association  with  perception 
and  cognition  (most  of  language  learning  during  the  age  2 to  7);  (3)  similarly, 
cognitive  models  are  learned  independently  from  language,  when  perception  signal 
data  are  encountered  for  the  first  time  without  association  with  linguistic  data.  In  (2) 
and  (3)  above  it  is  important  to  emphasize  that  cognitive  and  language  learning 
enhance  each  other.  The  original,  inborn  models  are  fuzzy  structures  equally  and 
poorly  matching  any  sensory  or  language  data.  In  the  process  of  learning  fuzziness 
decreases,  crisp  models  get  associated  with  specific  situations  and  phrases,  and 
cognitive  models  always  remain  associated  with  language  models.  Due  to  the 
integrated  (cognitive,  language)-model  structures,  association  between  language  and 
cognition  begins  at  a “pre-conceptual”  fuzzy  level,  inaccessible  to  consciousness. 
Similarly  a child  learns  a large  number  of  language  models,  which  association  with 
real  life  is  very  fuzzy;  throughout  later  life  they  facilitate  learning  of  corresponding 
cognitive  models;  similarly,  cognitive  (say  visual)  models  facilitate  learning  of 
language  models;  eventually  hi  and  h2,  cognitive  and  language  models  are  properly 
associated  (that  is  similar  across  the  system,  so  that  people  and  computers  understand 
each  other). 

5 Conclusion 

At  the  beginning  of  this  paper  I summarized  some  justifications  for  following 
biological  examples  in  engineering  system  design.  Still,  often  one  can  hear  a question: 
Why  does  an  engineer  need  to  know  about  concepts  and  emotions?  After 
mathematical  equations  are  derived,  why  not  just  use  them  for  developing  computer 
code,  why  should  an  engineer  be  concerned  with  interpretations  of  these  equations  in 
terms  of  instincts  and  emotions?  This  question  is  profound  and  an  answer  can  be 
found  in  history  of  science  and  engineering.  Newtonian  laws  can  be  written  in  few 
lines,  but  an  engineering  manager  cannot  hand  these  few  lines  to  a young  engineer 
and  ask  to  design  an  airplane  or  rocket.  Similarly,  Maxwell’s  equations  contain  the 
main  principles  of  radar  and  communication,  but  radars  and  communication  systems 
cannot  be  built  without  knowledge  of  electromagnetic  phenomenology.  For  the  same 
reason,  MFT  and  dynamic  logic  equations  need  to  be  supplemented  by  understanding 
phenomenology  of  the  mind  signal  processing  to  be  efficiently  applied  to  design  of 
high  level  fusion  systems.  For  this  reason  in  conclusion  of  this  paper  we  summarize 
the  main  aspects  of  working  of  the  mind  as  described  by  the  equations  given  in  this 
paper. 

Equations  in  section  4 describe  elementary  processes  of  perception  or  cognition,  in 
which  a number  of  model-concepts  compete  for  incoming  signals,  model-concepts  are 
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modified  and  new  ones  are  formed,  and  eventually,  more  or  less  definite  conneetions 
[high  values  of  f(hln),  close  to  1]  are  established  among  signal  subsets  on  the  one 
hand  and  some  model-concepts  on  the  other,  accomplishing  perception  and  cognition. 

A salient  mathematical  property  of  this  process  is  the  correspondence  between 
uncertainty  in  models  and  fuzziness  in  associations  f(hln).  In  perception,  as  long  as 
model  parameters  do  not  correspond  to  actual  objects,  there  is  no  match  between 
models  and  signals;  many  models  poorly  match  many  objects,  and  associations 
remain  fuzzy  (between  0 and  1).  Eventually,  one  model  (h')  wins  a competition  for  a 
subset  {n’}  of  input  signals  X(n).  In  other  words,  a subset  of  data  is  recognized  as  a 
specific  object  (concept).  Upon  convergence,  the  entire  set  of  input  signals  {n}  is 
divided  into  subsets,  each  associated  with  one  model-object,  uncertainties  become 
small,  and  fuzzy  concept-models  become  crisp  concepts.  The  general  mathematical 
laws  of  cognition  and  perception  are  similar  and  constitute  a basic  principle  of  the 
mind  organization.  Kant  was  the  first  one  to  propose  that  the  mind  functioning 
involves  three  basic  abilities;  Pure  Reason  (concept-models).  Judgment  (emotional 
measure  of  correspondence  between  models  and  input  signals),  and  Practical  Reason 
(behavior;  we  only  considered  here  the  behavior  of  adaptation  and  learning) 
[26,27,28].  We  now  briefly  discuss  relationships  between  the  MET  theory  and 
concepts  of  mind  originated  in  psychology,  philosophy,  linguistics,  aesthetics,  neuro- 
physiology, neural  networks,  artificial  intelligence,  pattern  recognition,  and  intelligent 
systems. 

A thought-process  or  cognition  involves  a number  of  sub-processes  and  attributes, 
including  internal  representations  and  their  manipulation,  attention,  memory,  concept 
formation,  knowledge,  generalization,  recognition,  understanding,  meaning, 
prediction,  imagination,  intuition,  emotion,  decisions,  reasoning,  goals,  behavior, 
conscious  and  unconscious  [5,8].  We  discuss  how  these  processes  are  described  by 
MET. 

A “minimal”  subset  of  these  processes,  an  elementary  thought-process,  has  to 
involve  mechanisms  for  afferent  and  efferent  signals  [8],  in  other  words,  bottom-up 
and  top-down  signals.  According  to  Carpenter  and  Grossberg  [29]  every  recognition 
and  concept  formation  process  involves  a “resonance”  between  these  two  types  of 
signals.  In  MET,  at  every  level  in  a hierarchy  the  afferent  signals  are  represented  by 
the  input  signal  field  X,  and  the  efferent  signals  are  represented  by  the  modeling  fields 
Mh;  resonances  correspond  to  high  similarity  values  l(nlh)  for  some  subsets  of  [n] 
that  are  “recognized”  as  concepts  (or  objects).  The  mechanism  leading  to  the 
resonances  between  incoming  signals  and  internal  representations  is  given  by 
equations  in  section  4. 

A description  of  the  workings  of  the  mind  as  given  by  the  MET  dynamics  was  first 
provided  by  Aristotle  [30],  describing  cognition  as  a learning  process  in  which  an  a 
priori  form-as-potentiality  (fuzzy  model)  meets  matter  (sensory  signals)  and  becomes 
a form-as-actuality  (a  logical  concept).  Jung  suggested  that  conscious  concepts  are 
developed  by  the  mind  based  on  genetically  inherited  structures,  archetypes,  which 
are  inaccessible  to  consciousness  [31]  and  Grossberg  [8]  suggested  that  only  signals 
and  models  attaining  a resonant  state  (that  is,  signals  matching  models)  reach 
consciousness.  Euzzy  uncertain  models  are  less  accessible  to  consciousness,  whereas 
more  crisp  and  certain  models  are  better  accessible  to  consciousness. 
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Recognizing  objects  in  the  environment  and  understanding  their  meaning  is  so 
important  for  human  evolutionary  success  that  an  instinct  has  evolved  for  learning 
and  improving  concept-models.  This  instinct  (for  knowledge  and  learning)  is 
described  in  MFT  by  maximization  of  similarity  between  the  models  and  the  world, 
eq.  (1).  Emotions  related  to  satisfaction-dissatisfaction  of  this  instinct  we  perceive  as 
harmony-disharmony  (between  our  understanding  of  how  things  ought  to  be  and  how 
they  actually  are  in  the  surrounding  world).  Since  Kant  [27],  emotions  that  are  not 
related  directly  to  bodily  needs  are  called  aesthetic  emotions.  Aesthetic  emotions  in 
MFT  correspond  to  changes  in  the  knowledge  instinct  (1).  The  mathematical  basis  for 
the  theorem  in  section  4 can  be  interpreted  psychologically:  during  dynamic  logic 
iterations  the  aesthetic  emotion  is  always  positive.  MFT  system  ‘enjoys’  learning. 

Signs  and  symbols  are  essential  for  the  workings  of  the  human  mind,  as  well  as  for 
accumulation  and  transmission  of  knowledge  in  human  culture.  They  are  also  used 
extensively  in  intelligent  and  multi-level  fusion  systems.  Scientific  theories  of  signs 
and  symbols,  however,  are  not  well  developed,  and  even  the  exact  meaning  of  these 
words  is  often  confused.  According  to  [32],  “symbol”  is  the  most  misused  word.  We 
use  this  word  in  trivial  cases  referring,  say,  to  traffic  signs  and  in  the  most  profound 
cases  of  cultural  and  religious  symbols.  In  mathematics  and  in  “Symbolic  AI”  there  is 
no  difference  between  signs  and  symbols.  Both  are  considered  to  be  notations, 
arbitrary  non-adaptive  entities  with  axiomatically  fixed  meaning.  This  non- 
differentiation is  a “hangover”  from  an  old  superstition  that  logic  describes  mind,  a 
direction  in  mathematics  and  logical  philosophy  that  can  be  traced  through  the  works 
of  Frege,  Hilbert,  Russell,  to  its  bitter  end  in  Gddel  theory,  and  its  revival  during  the 
1960s  and  1970s  in  artificial  intelligence.  Profound  use  of  the  word  “symbol”  in 
general  culture,  according  to  Jung,  is  related  to  symbols  being  psychological 
processes  of  sign  interpretation.  Jung  emphasized  that  symbol-processes  connect  the 
conscious  and  unconscious  [31].  Pribram  wrote  of  symbols  as  adaptive,  context- 
sensitive  signals  in  the  brain,  whereas  signs  he  identified  with  less  adaptive  and 
relatively  context-insensitive  neural  signals  [33].  Deacon  [32]  thought  that  the  essence 
of  the  human  symbolic  ability  is  two  interacting  parallel  hierarchies,  like  described  in 
section  4 hierarchy  of  cognitive  models  and  a hierarchy  of  sign  (language)  models;  he 
called  it  symbolic  reference. 

Combining  mathematical  developments  in  sections  4 with  the  above  discussion,  we 
reach  the  following  conclusion  for  consistent  meanings  of  signs  and  symbols.  The 
essence  of  a sign  is  that  it  is  an  arbitrary  notation,  which  can  be  interpreted  by  our 
mind  or  by  an  intelligent  system  to  refer  to  something  else,  to  an  object  or  situation. 
Symbols  are  psychological  processes  of  sign  interpretation,  they  are  equivalent  to 
elementary  thought  processes,  and  they  integrate  unconscious  (fuzzy  models)  with 
conscious  (crisp  models).  A simple  symbol  process  is  mathematically  described  by  a 
single  MFT  level,  like  in  section  4.  A complex  symbol-process  of  cognition  of 
culturally  important  concepts  may  take  hundreds  of  years  in  human  societies  and 
many  generations  of  MFT  agents;  it  may  involve  multiple  levels  of  MFT  or  the  mind 
hierarchy.  Future  sensor- webs  will  be  designed  using  this  biological  knowledge.  They 
will  participate  in  human-computer  collaborative  networks.  They  will  evolve  through 
generations  of  agents.  They  will  integrate  the  learning  of  language  with  the  learning 
of  complex  cognitive  concepts.  They  will  integrate  communication  with  sensor  signal 
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processing,  and  instead  of  quick  obsolescence,  their  performance  will  improve  with 
time  and  experience  by  accumulating  knowledge  similar  to  human  cultures. 
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